Field of the Invention
The present invention relates generally to multimedia data stream processing and, more particularly, to apparatus and method for processing digitally encoded multimedia data streams.
A multimedia data stream refers to a binary stream that represents multimedia data, by which is meant video, opional audio streams, and optional other streams (such as subtitles, private data, etc.).
A digitally encoded multimedia data stream refers to a compressed version of a multimedia data stream. MPEG-2 is an example of a standard that defines compressed multimedia data formats. A multimedia data stream that has been formatted according to the MPEG-2 standard is referred to as a "MPEG-2 encoded data stream".
The term "transport stream" is defined in ISO/IEC 13818-1:1996(e) page ix. The term "single program transport stream" (SPTS) is defined in ISO/IEC 13818-1:1996(E) page xi. A single program transport stream can also be abbreviated as SPTS; which is an example of a "MPEG-2 encoded data stream".
MPEG-2 is one particular system of digitally encoding multimedia data streams, and since this system is regarded as an industry standard by many, it will be used in the following detailed description of the present invention. MPEG-2 prescribes specific protocols and capabilities, while allowing essentially unrestricted ways of implementing the prescribed protocols.
MPEG-2 encoded data streams are constructed so as not to exceed any of the limitations of the decoder buffer. Splicing involves appending a new data stream to a selected point in an existing data stream, thereby replacing the existing data stream at the splice point.
The MPEG-2 system of digitally encoding data streams usually involves a specification that has been regarded as an industry standard and, among other features, its encoding and decoding specifications prescribe an array of data stream format, timing, synchronization, and device capability parameters essentially independent of hardware and software system implementations.
However, the MPEG-2 specification, as is the specification of other systems, is deficient with regard to applications and contingencies that were unforeseen at the time the specification was first proposed. For example, it failed to anticipate the growing requirements for digitally encoded data stream splicing support with standard encoders.
The term "existing data stream" refers to an old data stream, which is a present data stream. A stream replacing the existing data stream is referred to as a "new data stream", which is the stream that plays after a splice in.
A digitally encoded data stream is encoded such that, during decoding, a decoder buffer will continuously contain a sufficient number of frames for continuous decoding but not so many frames as to overflow the decoder buffer. Such a data stream contains a variable amount of reference data for recreating a video image.
Over time, the buffer levels in a decoder will fluctuate depending on which frames and frame types have been transmitted to the buffer and which frames have been processed for display by the decoder.
Typically, video frames of a digitally encoded data stream are received by a decoder in a different order from the order they are displayed. Decoders are required to buffer data streams so that video frames can be reconstructed and re-ordered readily.
Non-video data typically consists of audio data and other data private to the stream. Since audio is decoded separately from video, the time base of an audio packet can vary widely with respect to the time base in nearby video packets multiplexed on the same transport stream. Also, an audio buffer in a decoder is small relative to the video buffer.
Consequently, if the video and audio frames have similar time stamps, video is sent much earlier (e.g., about 100 milliseconds) before the audio. For this reason, the ideal splice-in and the ideal splice-out points with respect to video do not necessarily address the splice-in and splice-out points relative to non-video data. If audio is not spliced homogeneously with video, audio packets from the old data stream may be played with video from the new stream, or audio from the new stream may be played with video from the old stream.
Standardization efforts proposed by the Society of Motion Pictures and television suggest abandonment of conventional splicing methodologies in favor of a new extension to the MPEG-2 specification. The proposed standard contains a new integrated protocol directed specifically at performing seamless splicing of a data stream pair.
The proposed standard provides for incorporation of splice points and related splicing support information directly into the data stream at the time of encoding. Unfortunately, the proposed standard will require the expense of at least replacing a very large number of existing encoders in production today. Moreover, the proposed standard does not address splicing at arbitrary frames within an MPEG-2 encoded data stream.
Methods have been suggested recently for seamlessly splicing MPEG-2 digitally encoded data streams using the large install base of encoders without requiring new encoding protocols. However, these methods do not specifically address how to preserve the seam between two data streams for the audio and non-video components of the data stream.
Therefore, the problem persists for splicing two data streams while preserving the seam between them. Clearly, what is needed is a new and completely different approach to solving the problem, such a solution is provided by the present invention, which is described in detail hereinafter.